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ABSTRACT 

Experiments were performed at the Johnson Space Center (JSC), 
Manipulator Development Facility using the full scale Shuttle Remote 
Manipulator System (SRMS) to evaluate the effect of visual presentation 
through perspective display of the orthogonal forces and torques sensed 
at the manipulator end effector. The experiments investigated the 
effect of the display information on the management of forces and 
torques generated during payload berthing and deployment, as well as 
simulated satellite module change-out operations. The evaluation also 
addressed (i) issues of display format, including: force/torque scaling, 
point of resolution, and display mixing with video generated imagery, 
and (ii) task related variables of payload size, alternative sources of 
guidance information, and control mode. 

This paper briefly presents the results of a first-pass informal 
analysis of the analog, strip chart-recorded data from these evaluation 
tests. The results provide a relative measure of improvement in force 
management through the use of such a display, as well as information 
regarding the impact of display variables and task demands on operator 
performance. 

1 .0 INTRODUCTION 

Two experiments were performed at the JSC Manipulator Development 
Facility using the full-scale Shuttle RMS and the JPL two hundred pound 
range force/torque (F/T) sensor, four-claw end effector, and a 
perspective visual display of the forces and torques sensed at the end- 
effector. The equipment used in these tests, with the exception of the 
perspective display system, are described in a previous evaluation 
report by Bejczy and co-workers (1982). 

The two evaluation sessions provided an assessment of the effect of 
the F/T sensor and display system on SRMS performance. The first 
session investigated operator handling in large payload befthing. The 
second session dealt with small tool handling and simulated module 
change-out performance. Figure 1 provides a plan view of the payloads, 
their size, and location for the tests, in relation to the Rockwell 
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Shuttle point of reference (FOR), i.e., 236 in. forward of and i|00 in. 
below the orbiter nose point. 

The evaluation tasks were performed by four JSC personnel who were 
trained and MDF qualified in the use of the shuttle RMS simulator. The 
tests were performed in two sessions each of one week duration and 
separated by a six month hiatus. 

1 • 1 Display Characteristics 

The characteristics of the display format used for these evaluation 
are presented here. Since the time of these tests, we have made 
substantial progess in creating a three dimensional perspective display. 
This display technique is described in the final section of this paper 
on future research efforts. 

(i) The display, pictured in Figure Two, presents force and torque 
as filling from the center of the six axis perspective frame. The point 
of reference for the axes can be manipulated in software to correspond 
to the control reference frame of the operator, or any other reference 
frame deemed appropriate to the task. In the case of the PFTA payload, 
the X axis relates to the fore/aft axis of the orbiter, the Z axis 
refers to the elevation in and out of the payload bay, and the Y axis 
designates port/starboard across the payload bay. The torques about 
these axes are designated by filling of the pitch, roll, and yaw frames 
associated with each of the torques. In the case of the tool handling 
and module change out procedures, the display is referenced to the end 
effector and sensor reference frame as illustrated in Figure 36. 

(ii) The display provides force and torque readings to the 
operator referenced to the point of resolution (FOR) of the PFTA 
payload, in the first evaluation, and referenced to the sensor reference 
frame in the second evaluation. (The FOR can be varied through software 
manipulation of the data provided by the sensor system and can be 
calculated for the desired operator perspective, dependent on payload 
geometry.) The FOR chosen for the large payload berthing was the center 
of geometry of the payload. This FOR is forward of the center of mass 
of the payload to compensate for the small residual frictional forces 
associated with the payload counterweight system. The MDF counterweight 
system serves to simulate zero gravity operation for high mass payloads, 
such as the PFTA. 

(iii) The "sense** of the displayed forces shows the effect of the 
operator's control input on the payload. For example, in the case of 
PFTA manipulation, a roll to port that generates contact forces with the 
\port trunions is displayed as an increased torque to port and an 
increased Z force. The corrective control action to reduce these forces 
and torques is to roll starboard, i.e., the operator acts as if to push 
the extending display bar to zero, the center point. Operators 
generally found this "fly to" arrangement intuitive. However, when the 
payload is viewed from the aft cameras the sense of the display in terms 
of required corrective action is reversed. This caused some confusion. 
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and argues for a display reference that is dynamically referenced to the 
point of regard of the operator. Experiments and software requirements 
for such transformations are currently under consideration by the 
authors. 

(iv) Force/Torque display scaling proved sensitive to the payload 
geometry. Because of the large moment arm of the PFTA payload, torques 
generated at the bay trunions saturated the torque scaling more quickly 
than forces about the FOR. Software decoupling and rescaling of the 
torque display was accomplished, but there is some danger in this 
approach, in that sensor saturation may not bear a clear relation to 
display saturation. Future work will seek to provide both sensor and 
display saturation scales to the operator. 

(v) The display size could be reduced to allow split screen mixing 
with an operator selected camera view of the payload. 

1 .2 Performance Data 

The data collected were (i) total task time, defined as operator 
control initiation to payload berthed and latched condition, (ii) 
analog chart recording of the forces and torques sensed about three 
orthogonal force and three orthogonal torque axes of the sensor POR 
during the berthing operation, and (iii) digital recording of these 
forces and torques. In this preliminary evalution, statistical analysis 
is precluded by the large number of treatment conditions in relation to 
the number of data points gathered in the analysis. The evaluation was 
designed to survey the relative impact of the provision of and the 
format of visual F/T feedback, rather than to establish statistically 
robust parameterization of that effect. 

2.0 EVALUATION PROTOCOL 

2.1 Large Payload. (PFTAJ, Berthing 

Task; 

The performance required for this evaluation involved berthing the 
PFTA payload after it was deployed to a random position above the paylad 
bay trunion guides. The task represents the precision placement portion 
of a payload berthing task. The berthing task was performed ten times 
by each subject after familiarization and briefing runs on the display 
characteristics. The ten test trials were performed under varied 
feedback and control conditions as illustrated in Table I. The control 
point of reference for these tests was the orbiter control mode, in 
which the operator controls the end effector of the RMS in relation to 
the shuttle body. Translation axes of the two-handed controller refer 
to for/aft, port/starboard, and elevation in/out of the bay. Rotational 
axes of pitch, roll and yaw are referenced to these trannslational axes. 
(The control mode for the majority of the tests was a resolved rate 
control. The exception to this was a joint by joint control mode which 
had its greatest impact in dramatically increasing required performance 
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for PFTA Berthing Experiments 


26.7 




time for all operations.) 

Results: 

A very general discussion of results is presented for this task. 
Analysis of the digital data, as opposed to the analog chart recording, 
is being pursued with the intent to describe the effect of the varied 
feedback views in conjunction with the F/T display. At this point we 
will confine our discussion to the management of forces and torques with 
and without the visual display from the sensor. 

(i) Force/Torque generation: 

- Provision of force/torque information via the visual display 
reduced the loads on the PFTA payloads and payload guides during 
berthing by 30-50? of the values generated without the provision of the 
display. 

- For those forces generated in excess of 50? of the dynamic range 
of the sensor, visual display of the force/torque values reduce the 
duration of the application of that excessive force by 60-80?. 

(ii) Task completion time: 

- Task completion time was most dependent on the individual 
operator's control strategy. The directions stressed both accuracy and 
speed in task completion; however, speed was consistently sacrificed to 
performance accuracy. 

- Provision of F/T information slightly increased the usual task 
completion time for a given operator. This was probably due to the 
requirement for shared attention between visual displays of payload 
position and the force/torque display. 

- Several operators noted that the provision of the F/T display 
expedited trajectory planning in the case of excessive force 
application. The F/T information could be used diagnostically to 
identify the cause of the problem and to provide a basis for replanning 
the maneuver. This was especially true in the case of keel trunion 
misalignment; because the source of such an error is not readily visual 
available. 

- As noted, the effect of the varied feedback conditions will be 
examined through analysis of the digital force/torque data. 

2.2 Tool use and Simulated module change out 

Task: 

The tool use and module change out task involved manipulation of 
the modules of the task board illustrated in Figure 2. The flat screw 
driver blade was used to unlatch the box module and replaced in the 
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appropriate receptical. The module was then grasped, removed and 
reinserted. The screw driver blade was then retrieved and used to latch 
the module back in place. The task was performed in the end effector 
control mode, in which the control and display was referenced to the end 
effector position, independent of its position in the shuttle bay. 
Figure 3b illustrates the coordinates of the end effector reference 
frame. Figure 1b illustrates the placement of the task box in relation 
to the shuttle bay. 

Results: 

It is significant to note that three of the four subjects were 
unable to compete the module extraction task without the provision of 
visual force and torque information. 

Several representative figures have been abstracted from the analog 
performance record to illustrate typical performance profiles. 

- Figure 4 shows the calibration scale for the data represented. 

- Figures 5a-5b shows the basic extraction/insertion sequence. The 
generation of excessive forces and torques in the absence of the F/T 
display is illustrated in 5a. In fact, the trial was aborted when the 
forces were sufficient to damage the module during the test. Successful 
completion of the same task sequence is demonstrated in 5b. 

- Figures 6a-6b provide a direct comparison of module insertion 
sequences with and without the F/T display. A comparison of 6a and 6b 
illustrates increased levels of force/torque generation and increased 
task completion time for the single subject who was able to complete the 
module change out in the absence of the F/T display. 

- Figure 7a provides a demonstration of a jam in which module 
extraction is aborted due to excessive force in the X and Y axis and 
torque about the Z axis. The diagnostic capability of the display is 
illustrated in Figure 7b, in which, despite the occasional generation of 
high force and torque values, the subject is able to successfully 
complete the module extraction. 

- Figure 8a-8b shows successful force management in the tool use 
sequence of the task as a function of the provision of the force/torque 
display. 

3.0 FUTURE RESEARCH EFFORTS 

One of the major concerns in the presentation of force/torque 
information is the speed vs. cognitive information transmission dilemma. 
In other words, it is the dilemma of trying to transfer to the operator 
as much information as fast as possible without having a degradation of 
performance. This information should be presented so that the operator 
can cognitively understand and utilize it. 
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Items for display improvement; 


i) The display should be as smooth as possible. The operator 
should be concentrating on the information present in the display, not 
on the display itself. 

Most computer graphics display hardware is display-bound. The more 
pixels and polygons drawn on the screen, the slower the pixel write 
speed. Since most hardware internal graphics subroutines (draw 
rectangles, draw circles) are faster than software generated 
subroutines, it is optimal to use as many hardware oriented commands as 
possible. 

ii) The display should present the information in a natural manner 
(i.e., true perspective view). 

iii) Color should be used to enhance contrast between different 
display parts. 

The true perspective 3-D Force/Torque display; 

We have been able to make progress in the development of real-time 
3-D displays because the substantial leap in the speed of current 
computer graphics hardware. The displays we used at JSC had a refresh 
rate of 4 to 5 hertz and there was a significant speed difference 
between the X/Y axis and the Z axis. With current display technology, a 
refresh rate of 30 hertz is easily achieved with much more true and 
complex display of forces and torques (Figure 9). 

The torques and forces are color and directional coded. Red 
indicates a negative force or torque and blue indicates a positive force 
or torque. The torques follow the right-hand rule around the force 
axis. The display is projected in true perspective. The box around the 
display enhances the perspective image. The reticular marks divide the 
force bars into quarters. These marks help the operator gauge force on 
each axis. This is true especially in the case of the negative z force 
axis. 


We thought about adding a grid on the bottom of the box to enhance 
the perspective image but it was decided that it would add too much 
clutter to the display. 

4.0 CONCLUSION 

In general, the operators considered the F/T display informative, 
and the data illustrate the fact that management of forces and torques 
improved when the display was used. In fact, the precision module 
extraction and tool use task was only able to be performed with the 
display aiding. There were a number of factors noted that could 
contribute to an improvement of the display format, and these have been 
the focus of our efforts in the development of the three dimensional 
perspective display. In particular, the following issues are being 
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Figure 9. New Force- Torque Display 
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addressed; 


1. The update rate of the display used in the evaluation was on the 
order of 4-5 Hz. While this was adequate for slowly moving payload 
operations with the large PFTA, there was a noticeable jumping in the 
display resulting from force generation with the smaller payloads. The 
new generation display has an update rate on the order of 30 Hz. 

2. Reticular marks along the frame axes have been added in the new 
display to give the operator more detailed information on the level of 
forces being generated in the range of the display scale. 

3. As noted, coordination of control, display, and point of regard 
reference frames is being investigated in an effort to maintain the 
operator’s situation and reduce disorientation in interpreting the 
operational effects of force generation. 

4. There is a great potential for the use of color to increase the 
information density of the display without adding clutter. Color coding 
of direction and magnitude of the force/torque vectors is being 
investigated in the new display development. 
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